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Ach!«ving A High Test Score in the Social Studies Ciassroom: 
How Student Success Depends Heavily Upon 
Unstated Criteria Used by the Teacher^ 

Robert J. Stahl 
George W. CMIooat 
Jerry Ligon 

INTRODUCTION AND CONTEXT 

Student success in social stucSes classrooms is depertdent on mucti rrwre than |ust paying 
attention, acquiring Information, and effort. Ttie quality of instmctlon. the pace of learning tasks, 
students' prior knowledge and entering abilities, extent and focus of student motivation, student 
perceptions of personal effiracy, and type and quaDty of learning activities are six factors that affect 
learning and contribute to each student's success (e.g., Bloom, 1976; Dempster. 1989: Carroll, 1989; 
Doyle. 1986; Slavin. 1989; Stahl. 1989. 1990a.b: 1982). Factors such as these influence what, how 
much, and how welt students ieam both with'n and outside social studies classrooms. While these 
factors influence what is ieam«i on a day-by-day, course-by-course, and student-by-student basis, 
the extent of success for each student is measured and largely determined by his or her performance 
on tests and academic tasks.^ 

Tests: Defl nittons and Selected Roles 

Ideally, within social studies classrooms, tests and academic tasks are directly afigned with 
targeted outcome knowledge and abilities described by curriculum &nd testier objectives. Tests 
should serve to focus and channel student attention, effort, and learning. At the same time, tests 
should provide opporturtities for teachers to gather Infonnation concerning the extent to which each 
student has attained or surpassed the knowledge and abilities expected (Airasian, 1991; Stahl. 
1989). In addition, tests and their evaluation affect students* feefii^s of competence and influence 
how students approach personal study (Crooks. 1988). While tests perform these arwl many other 

''The authors i^preciate the ecStorial assistance of Micheal Jackson, Mountain View High School, 
Mesa. AZ In preparing this report 

^is of course assumes tttat teachers detennine each student's final grade on actual performance 
rather than peroeived notk>n8 about the student's success. It also assumes th^ the studenTs grades 
are not arbHrari^ detemnlned oy "curving" grades and other artificial devices to inflate grades over 
actual performance meaaires. In this sUidy. a lest is "any device which is used or may be used to 
obtain Infonnation concerning the ability, concfitlon, status, or quality of an entity, perfonnance or 
phenomenon at a particular moment relative to one or mora criteria or standards' (Stahl, 1983. 1989). 
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roles, the scores artd O^dos students receive relative to their actual responses indicate how 
successful they have been in their teaming efforts. Conventional wisdom assumes that the higher the 
score or grade, the more successful the student has been -- and Is. Formal tests are clearly the major 
device secondary teachers use in order to measure and assess student success (e.g., GulUckson. 
1985). 

To obtain Information relative to student learning, teachers ask questions, assign students tasks 
to complete, and provide directions amf instnjctions for students to fulfill. Students are expected to 
use the information In the questions, assignments, and instmctions as clues to guide their responses 
in providing evidence of their thinking ai«J achievement, Stuc*erts should have a clear nolton of what 
they need to do or provide In order to do :>eir best. They took to cues in the questions to due them 
as to what is needed. However, what woukl the com^et^ences be for student success were the cues 
provkfed not defined to provkje a clear set of expectations for them in test situations? 

Clarity and preelfilon of expectations 

Extensive planning should result in every test Hem clearly infonning students as what they must 
provide to earn full credit for that Item. Cuing tenns should be clearly defined so that students and 
teachers aDke have a consistent framework for how answers are to be constructed and written for full 
credit. For instance, if the teacher asks "in what year was President J. F. Kennedy assassinated?," 
students are expected to provide the yea. In this instance, the 'nnnation in the question 

makes it quite dear what spedfto information students need to provide to earn full credit. Such 
predsion eliminates any ambiguity as to what the teacher is expecting and what students need to 
provide to be successful in that situation. At the same time, this predsion does not place students in 
the situation of having to guess )ust what infonnation the teacher wants and will accept for them to be 
successful. 

When given vague or ill-defined expectations, however, students must rely on various survival 
strategies to direct searches for useful cues in the questiors and instructions provided. Ambiguous 
dues may be poorly dedphered or ntissed altogether. At other times, such cues may lead students tr> 
responses quite different from what the teacher expects and what the student, with more predse 
cues, couki have provided. 

Unfortunately, the predston liwstrated In the above example Is contrary to what many students 
typically find on woiksheets. task assignments, tests and teacher generated instmctions. For 
Instance, were the teacher to asked, "When was Preskfent John F. Kennedy assassinatedr, would 
the correct answer be "19637"; "November, 1963?"; "November 22, 1963r: or "on a Friday 
aftenwon. November 22, 1963?" Suppose, however, a student answered "Before I was bom." "On 
the day he died," "In the 1960s." or "After World War II ended." While these are ail correct answers to 
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the question as it was acbiaRy asked (i.e.. worded), few teachers would iikety accept such statements 
as desenring (uH crecSt for an appropriate answer to this "when" question.^^ The typical teacher fuUy 
expects students to "know" exactly what inlonnation vviti and woni be accepted in situations where 
the "when" queston is aticed. That sometimes the $am teacher wants only a year, at other times a 
complete date, and at still other times infonnation such as time of day. a particular month, day of the 
week, or even period of one's Bfe. makes what is an acceptable answer to such questions very 
uncertain if not an absolute mystery for most students. 

Students frequerttly find themselves In situations where what information they need to provide in 
order to have a correct, conplete response Is r»t dear. The cue wi. -te teachers use in c^estlons and 
directions include terms such as "identify," "describe." "analyze." artfJ "explain" words that are 
fF8<^enUy used but rarefy defined predsefy in soda! stui^es classrooms. Uitiortunately, these words 
are only a sample of the many amt^guous cue temis that continually find themselves being used In 
dassroom situations. H a student is to sua»ed. he or she must often ignore the cues adually 
included in the question atKl rely iqson guesstimating corredty what the teacher most likely wants as 
an ^ropriate response at that moment. 

Students who find themselves in this "survival context" soon realize that their success is largely 
dependent upon how weB they can invent and constmd meaitings for the cue temis being used: how 
well they can retrieve ir^nnation and then oonstrud responses that are consistent with their invented 
meaning for the cue tenns; and how consistent their invented meanings are with those meanings 
their teachers win use in assessing the quafity of their responses.^ In such in^nces. their academic 
focus Is not direded toward demonstrating sodal stucfies achievement, but on inventing ways to be 
successful on that test. 

Academic success for students in the sodal studies presupposes that their teachers actually (a) 
have an expOcatable and predse meaning for each tenn and for the type of Information each terni 



3we have found that different teachers would give fun credit for each of these Afferent answers. For 
instance. vMle certain teachers gave full credit for "1963,' others gave only half or no cretfit for "1963" 
and fuD cretflt only for "November 22. 19^" for the exad same "when" question. 
^The use of this question that calls for the recaN of a spedfic date is used to illustrate the need for dear 
cues regarcfless of the Information being sought In the answer. This example In no way su^^sts that 
the authors erelorse this type of question as belr^ the most Important or suitable one for sodal 
stucfies dassrooms. 

SAnother , perhaps more vivid and relevant situation where this Invention occurs may provide a clearer 
Illustration for the In-sewlce edi^stor. A number of urtiversity students have told the researchers that 
during many of their urdversity courses they depend heavily upon the feedback their inslnKslors give 
after the first test In order to determine how thay need to answer test questions on all figure tests. 
They then hope their fnstnictors continue to use this same criteria for responses and evaluatli^ for an 
future tests. Comments by students such as "If I knew thaf s what he wanted for the answer to that 
questton . ..." and "The way her question was worded certainly ffidnl lead me to ving the kinds of 
InfonTHUon she accepted . . . ," reveal that the ambiguity mentioned here about predse cues and dear 
agreed-upon meaning for critical tenns used as cues In questions and Instmctions Is not limited to 
pre-coHege level classrooms. 
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requires In the responses students nuist provide, (b) i^ed this meartii^ to frame the test item; (c) wiO 
use this meaning to determiiw an »:ceptable fo^nse tor each particular Item: and (d) will 
communicate and/or negotiate clear mearrings tor these tenns so students can construct responses 
to meet the criteria the teacher wiN use. Teachers m^ provide these meanii^ on their own or work 
jointly with students to arrive at an acceptable set o< definitions. The extent to which the teacher tails 
to do any of these four increases the probat)ility that student success is detemtined arbitrarily by their 
teachers. In these Instances, the degree of success In the fomi of a score or grade Is dependent 
upon ambiguous, unshared, and often arbitrary criteria. Putting it more bluntly, in mnv dassrooms, 
what students pmvids as an «M»rg anti what their seofBS />i» mav hg vsrv inaccurate reoresg/tf affons of 
what thftv hav e actuate achiavad and shtniM reeeive a^ a score or grade. 

Social studies educators on all levels continue to use ilktefined amt usually an^^^us terms to 
label expected student processes, outcomes, and responses. These educators remain unaware or 
have ignored the problems generated for studems by unclear definitions for each of the abilities they 
re<^re students to exhibit. Part of the reason for this failure may be that there has not been sufficient 
evidence of the kintte of affects ambiguous tenninotogy may have on studem achievement. In the 
interviews we conducted, the majority of teachere did not befieve this lack of specificity generated any 
real problems for most students and that time spent developing, teaching, and using the definitions 
for these process and re^nse ter?Ti> would not resuB In any major changes In student responses. 

This study was conducted, In part, to investigate the extent to which a student's success on a 
written test would be affected by the use of unstated definitions aivf standards to assess their 
Individual performances. This report focuses on the responses of pre-servlce teachers In social 
studies methods In order to Illustrate how their own success on a test would be affected by the 
meartngs and standards they and a grader used to evaluate their responses. Hopefully. If teachers 
were to experience firsthand how the use of Ill-defined tenns and personalty-Invented meanings 
affected their own test scores, then they might comprehend what secondary students must deal with 
each time they respond to test Items that Incbjde these same tenns. This report represents the initial 
findings of an on-going study. 

Pumosas of this study 

The purposes of this research were: 

(a) to determine how pre-servtoe sodai studies teachers respond to test items that include 
commonly used but rarely defined cue terms senHng as guides or clues for framing appropriate test 
item responses; 

(b) to detemtine the extent to which pre-servlce social studies teachers agree on what the 
appropriate responses are to the identical set of test items; 
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(c) to investigate how pre-service sodal ^ies teachers define selected cue terms that are 
commonly used in test Items and task assignment relative to their measurement of student academic 
success in social stupes classrooms on the pre-co9ege level; 

(d) to investigate pre-service teachers' reactions to information concerning differences in 
definitions they held and for comnf^niy used but rarely defined cue terms. inctu(£r^ the perceptions 
of the probable implicaiions of these differences for their own teaching and tor student leamir^, test 
responses, and test scores. 

FHiipose b was also the focus of an initial investigation in this same area with a small volunteer 
population of secomiary school social studies students. A brief overview of the prefintinary findings 
from the responses of these students is provided. 

Rnaliy. possible in^lications of the sintilarWes and differences among teachers* dennitions for 
these cue terms and of the use of these definitions as criteria for success for stiidents enrolled In pre* 
college social studies courses were explored. 

PRESERVICE SOCIAL STUPtES TEACHERS 
Procedurfts and Methods 

Pre-sen^ce teachers enrolled in five undergraduate social studies methods courses at Arizona 
Stale University, Brigham Young University, and Eastern Illinois University volunteered to participate In 
this study. The separate classes were made up of either all elenwntary level or secondary level pre- 
sent teachers. 

Students in these classes were provided a xerox copy of the first part of Chapter 21 : 'Settling the 
Last Frontier (1865-1890)- from Triumph of the AmBriean Nation (Todd & Curti. 1^0: pages 486- 
496). They were also given a three page "Examinatton'' as a required take home, open-book test. 
ISee Appendix A for a copy of this exam.J The cue tenns used in framing the 22 exam Hems were 
selected from among a 6st of cue temns one of the researchers had collected over a five year period 
from an assortment of teacher, commercial publisher, and author developed tests, worksheets, 
discussion questions, and directions. Specific cue tenns included •Identify,* "describe,* "analyze," 
"explain," and "centre." The cue tenns "who" and "when" were also used in selected Items. 

The targeted cue tenns were not defined for any participant prior to Ns or her conptetion of the 
take-home examination. Participants were instnjcted that this take-home exam was to be completed 
and that the exam would be graded. They were directed to answer all questions as correctly and 
completely as possible so that, when the exam was graded, they would earn full credit for their 
responses. These students were Instructed to bring three copies of their completed assignment to 
class on the next day the class met. 



ERIC 



7 BEST COPY AVAILABLE 



(tehieving A H^h Test Score in the Soc^ Stud»6 Cbs«oom 



(Stahl. Chitooat. & Ligon, 1991) 



During the next dass penod. each partidpanl was given the anonynious exam papers of two of 
his or her peers In the sanie ciass. Once the papers were stributed, each partldpanj was asked to 
read the two exam papers and to "grade iT as though these were turned In by students In an 
elementary or secondary class they might teach. Participants were cfirecied to assign a number score 
of from "5" to "0" for eatiti of the 22 Items: a '5" was to be assigned to responses they considered to 
be correct and complete for each Item while a "0" meant that the response was totally Incorrect and 
unacceptable such that It did not merit any credits. Scores of between "0" and "5" were to be 
assigned for partially conect responses. No further details regarding points or standards of 
correctness were provided. Participants were also Informed that. H they wanted to do so, they could 
use their copies of the text chapter to look up Information relative to the test Items.^ Participants were 
then given time In dass to score each Hem and to calculate the total polrtts out of the 1 10 maximum 
each exam could earn. 

The 11 students In one methods class were also directed to assign a letter grade to each exam 
they had just evaluated and scored. This task ws& assigned after they had an computed total test 
scores for th j two exams. These students were directed to assign a letter grade of A, B ,C, D, or E 
and a "+" or "-" should one of these be appropriate. They were directed to assign the grade based 
upon the criteria they would use H the 3xam papers they received were from students In high school 
Classes they might teach. For data analysis purposes, these letter grades were converted to 
numerical equivalents. A value of 4.0 was assigned for an "A; "3.7 for an "A-:" 3.3 for a "B+f and so 
forth. This procedure would provide data as to how consistent these pre-service teachers would be in 
assigning sin^iar ^ades to similar scores. 

A total of 84 students In the five methods classes completed aU of the tasks. Papers of students 
who brought In unreadable exams (due either to their own handwriting or to the poor quality of their 
reproduced copies) or who failed either to complete the scoring exerdse or to record legible scores 
for all 22 test items were excluded from the data analysis part of the project. 

These procedures enabled the researchers to exantine the extent to which the responses of all 
participants varied among one ar^ther for each particular item and for the overall test. By having each 
paper evrluated by two IrKfivlduals, the dlff erer^jes In the criteria for an acceptable response between 
two separate graders for each Item tor the same test could be studied. 

Ih9 next section reports preliminary findings relative to the success these pre-service teachers 
attained on this take-home examination. 



^Nearty all participants were observed to have refen-ed back to the chapter at some time during the 
time set aside in the class period to comf^te the gracfing task. 
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Findinffs fef g«am Siiores of Thesi* Pfftscrvlcft Teachers 

An Initial set of 2 ' variables was generated. For each of tfie 22 test Items, a sc»re of 0-5 was 
recorded. A separate wtat score for Items 1-20 (for a total of 100 points) and for all 22 items (I.e.. 110 
points for the total test score) were computed because of the possible responses that may have been 
generated In reaction to Items 21 and 22,7 ghe reason for this possibility will be elaborated later.) 
Scores for each of these variaWes were available lor each exam as graded by the two graders. For the 
1 1 teachers In one dass. a numerical equi^lenl to a tetter grade was general6d as a 25th variable. 

Tabie 1 provides a sample of the actual responses to two of the 22 items t*y randomly selected 
participants. The numbers In the parentheses foUowIng each response in Table 1 represent the 
scores assigned to that particular respor^ by each of the tvw> peer "graders " For instance. I' a 3){5) 
occurs, one grader assessed this resporee as deserving 3 of 5 points while the second peer grader 
assessed it as deserving all 5 points. {See Appendix B for samples of iTK>re student responses to alt 
22 Items.] 



Table 1 about here 



Table 2 lists the mean, standard deviation, and maximum scores for each of the 24 variables for 
these 84 pre-service teachers {See Appendix C for more data related to these scores.). These data 
include the scores on both copies of each partldpanl's exams. For test Items 1 - 22, the maximum 
score was a "5." A maximum score of "100" was possible for the SUM20 variable (I.e.. sum of Hems 1 
through 20) and "1 10" was possible for SUM22 (I.e.. sum of Items 1 through 22). Given the nature of 
the assignment and that It was a take home exam that wouW be graded, the variation In scores tor each 
item ar^ for the total test score Is quite remart(able. 



Table 2 about here 



The mean scoie for the 168 exams graded (i.e„ two scores per 84 participants) was 94.2 (s.d. « 
13.1) of a maximum of 1 10 or 85.7% correct among these preservlce teachers on this exam. 



^Test Item 21 asked students to provide information that wou\6 indicate that they had an 
"understanding" of the Information in the chapter. Item 22 asked them to reveal that they had 
"conprehended" this same Infonnation. Because of the Bkefihood that some students would claim 
that there was no difference between "con^ehension* and "understanding " the researchers 
decided to compute two separate lotal scores." One such score would be the total of the 20 Items 
excluding items 21 and 22: the secorti score was the sum of the 22 Items. The tables Included in this 
paper include data relative to both of these totals. However, unless Indicated othenwise. any 
reference to a "total test score" In the narrative refers fust to the sum of the 22 Items. 
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Table 1. 

A sample of the responses of social studies methods students to and the scores of the two peer 
graders on two Herns on the 22-item take home examination. 

ytfhf^ waft ChiPf Joseph? 

a) He was the leader of the Nez Perce Indians. (5}{3) 

b) Leader of the Nez Perce Indians. Led hte people away from their homeland. Surrendered to the whhe man. 

c) Leader of the Nez Perce buJians whose tribe, after trying to run to the Canadian border, vims surrounded by 
American troops who wantmj to relocate them w a resenralion outside the tortBe Great Plains. (4){5) 

d) Chief Joseph as the leader of the Nez PioroB Indians, a Pacific Northwest tribe. (3)(3) 

e) Leader of a group of Indians known as the "Nez Perce.* They were a peaceful tribe who had Hved where the 
present day stales of Oregon. Washington, and Idaho meet. Chief Joseph surrendered to the U.S. Arniy in 
1877 In Momana near the Canmiian border. His people were freezing to death. Chief Joseph had been 
attempting to lead hs peofto to a free land in Canada. (S)(5) 

f ) Leader of the Nez Perce Indians w)» choose to ran rather than fight the U.S. caveriy. (5)(3) 

g) Chief Joseph was the leader of the Nez Perce Indians who h»d lived on the Great Plains until pioneers 
Invaded the land. He b^me famous for his attempt to fie© to Canada and ultimalely surrendered to U.S. 
troops very dose to the Canadian txtrder. (5)(4) 

h) Chief Joseph was the leader of the Nez Perce Indians in the 1 870s. He attempted to lead his people to a new 
homeland in Canada when the United Stales government ordered them out of their original lands. (5)(3) 

Describfl: Chief Smi no BulL 

a) The leader of the Chevenne Indians he was described as honest, able, and idealistic. (1 ){5) 

b) Able, honest, and idealistte Sioux loader and chwf. Led the Sioux over Custer In the Battle of the Little Big 
Horn. (5)(5) 

c) Able, honest, idealistk:. I assume he's a-;'}yenne. but not sure. {4)(3) 

d) Chief Sitting Bull was the teader d the Ch^enne warriors. He was aWe. honest and Realistic. (5){4) 

e) He was the able and iwxasi and Uealistk: teader of the Cheyenne warrbrs who fought Custer at the Battle of 
the Little Big Horn in Mb»Ytanain1876. (5^3) 

f) Able, honest. kieaiistlCL (5)(4) 

g) Has been descrO^ed as an outstanding warrtor conskiered to be honest and Idealtetic who akfed in the defeat 
of Custer at Little Big Ktom but %vho eventually was defeated by the U.S. Cavalry tmopa. (S)(5) 

h) Sitting Bull was an able, honest and idealistic leader of the Sioux Indians, one of the two leaders who played 
a major role in the defeat of General Cu^er at the Little Big Horn. (5)(5) 
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Table 2. 

Mean, standard deviation, and maximum scores assigned by the two peer graders for the in(Svlduai 
items are] total test for the 84 preseivice sodal studies teachers in the five methocte classes ' 
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4.514 




«#*vv 


0 11 










0 Ofi 


Q13 


4.37 


1.31 


5.00 


0.10 


014 


3.98 


1.64 


5.00 


0.13 


015 


4.63 


0.81 


5.00 


0.06 


016 


4.72 


0.71 


5.00 


0.05 


017 


3.74 


1.67 


5.00 


0.13 


018 


4,45 


1.15 


5.00 


0.09 


019 


4.36 


0.88 


5.00 


0.07 


O20 


4.35 


1.07 


5.00 


0.08 


021 


4.27 


1.36 


5.00 


0.11 


022 


3.10 


2.10 


5.00 


0.16 


SUM20 


86.86 


11.50 


100.00 


0.89 


SUM22 


94.23 


13.10 


110.00 


1.01 



•The total nun^jer of scores for each variable was 168. representing two scores for each of the 84 
participants. 
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A t-test. used to determine whether the scores tor the Identical Itenis by the two graders were 
essentially the sanw, found a statistically signHicant tffference (p < -001) in the scores assigned by the 
two peer grieis tor ail 28 variables examined (see Table 3). 



Table 3 about here 



The mean GPA assigned to the scores by the 1 1 teachers In the one class was 3.27 (s.d. « .63). 
with the lowest grade assigned being a "C" attached to a score of 70 of 110 points (63.6%) and an 
"A-t-" assigned to a highest score of 106 (96.4%). Table 4 Ksts the scores assigned in this class by the 
peer graders and the letter grade given to those scores. The mean score for the 22 exams evaluated 
in this dass was 90.9 (s.d. - 9.29) or 82.7% of the 110 points possible. 



Table 4 about here 



The extent to which the difference between the two overall test scores for the same panicipant's 
exam (i.e.. sum of Items 1-22) accurately reflect or actually misrepresent the real differences In scores 
across the 22 test items for the two graded copies of each student's exam was Investigated To find 
out, the absolute difference between the two scores for each test item for the sanro exam was 
calculated. For instance, one grader may have assigned a score of "3" to the answer to the first t^st 
Hem and a "4" to the second, ly^eanwhile, the second peer grader may have assigned a "5" to the first 
test answer and a "i" to the second item on the Identical exam. Note that there was an absolute 
difference of "2" points (3- 5 = -2) between the two scores for the first test Item and "1" point (4 - 1 » 3) 
for the second Item. All negative numbers (e.g., 3 - 5 » -2) were converted into positive numbers 
(e.g.. the -2 becomes +2). This procedure allowed us to calculated the total amount of the differences 
for each test Item and the 22 Items combined. The maximum absolute difference possible for each 
test Item was 5 points. Table 5 lists the mean, standard deviation, and range for the absolute 
difference between the two sets of scores for each of the 22-ltems and for the total test score for the 
84 students.® 



Table 5 about here 



^The variable DSUM22 represents the difference between the total test scores for all 22 Wem by the 
two peer graders. The variable S{MA6DF22 Is the <fifference between the total of the absolute 
differences between the two graders' scores for each of the 22 itenrts for the same exam. DSUM20 
and SMABDF20 are variables referring to sums for Items 1 through 20 as mentioned previously. 
DGPA Is the variable for the difference In the rwmerfcal equivalents to the letter grades assigned by 
the peer graders. 
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Table?. 

t-vatues for (fifferences between separate scores as^gned by two gi-aders to itienlicai answers for 22- 
individuaf test iterr^ and total test for the 84 presenHce social studies teaOwrs. 



X/AOIADi C 


1 VCIHI09 


PR>rn 




5 97 


0 0001 

V • W W ff 




7 ftl 

/•Of 


0 0001 




A ft') 

0«0w 


0 0001 


r\A 




0 0001 

V'VWW 1 


UO 


7 IR 
/-ID 


0 0001 


Oft 


7 QQ 


0 0001 




7 R1 


0 0001 




ft '^Q 


0 0001 


no 


7 OS 


0 0001 


W • V 


ft 1Q 

D« 1 if 


0 0001 


W 1 1 


7 Aft 


0 0001 


W 1 (& 


5 P7 


0 0001 


W 1 M 


S AO 


0 0001 


w t ^ 


A 93 


0.0001 


Vr 1 P 




0.0001 


W i o 


5 A6 


0.0001 


017 


7.33 


0.0001 


018 


5.99 


0.0001 


Q19 


6.24 


0.0001 


Q20 


6.67 


0.0001 


021 


5.09 


0.0001 


022 


6.60 


0.0001 


SUM20 


11.01 


0.0001 


SUM22 


11.44 


0.O001 


SMABDF20 


16.50 


0.0001 


SMABDF22 


16.65 


0.0001 


DFABTO20 


8 66 


0.0001 


DFABT022 


9.36 


0.0001 
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Table 4. 

The total test scores, percent of total test score, and letter grades assigned to the test scores by the 
1 1 preservice teachers in one methods class. 



TOTAL TEST LETTER 

-SQQS reg&ff fflftDL- 

106 96.4 A-j- 

103 93.6 A 

102 92.7 A- 

100 90.9 A 

99 90 0 A- 

95 86.4 A- 

95 86.4 B+ 

94 85.5 B 

34 85.5 A 

93 84.5 A- 

93 84.5 B 

92 83.6 

92 83.6 64 

91 82.7 B 

SC 81.8 B 

90 81.8 B 

85 77.3 B 

83 75.5 B+ 

82 74.5 B 

78 70.9 C 

73 66.4 C+ 

70 63.6 C 
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Tables. 

Mean, standard devlaJlon, and maxlnium abRoiute diwerence between two scores assigned to ttie 
same examination bv two pee: graders for the 84 preservice social studies teachers In the five 
methods classes. 



VARIABLE 


HEKH 
DIFFERENCE 


STANDARD 
DEVIATION 


MAXIMUM 
DIFFERENCE 


STD ERROR 
OF MEAN 


DQ1 


0.61 


0.93 


3.00 


0.10 


DQ2 


0.63 


0.74 


3.00 


0.08 


DQ3 


0.55 


0.73 


3.00 


0.08 


DQ4 


0.65 


1.08 


5.00 


0.1 


DOS 


0.55 


0.70 


2.00 


0.08 


DQ6 


0.68 


0.78 


3.00 


0.08 


DQ7 


0.64 


0.75 


3.00 


0.08 


0Q8 


0.75 


0.62 


3.00 


0.09 


DQ9 

W 1MB fir 


0.88 


1.14 


5.00 


0.12 


DO10 


0.86 


0.96 


3.00 


0.10 




0.95 


1.14 


5.00 


0,12 


DC12 


0,52 


0.91 


4.00 


0.10 


DQ13 


0.52 


0.83 


4.00 


0.09 


D014 


1.17 


1.54 


5.00 


0.17 


DQ15 


0.54 


0.88 


4.00 


0.10 


DQ16 


0.46 


0.75 


3.00 


0.08 


DQ17 


0.88 


1.10 


5.00 


0.12 


D018 


0.51 


0.78 


4.00 


0.09 


DQ19 


0.61 


0.89 


5.00 


0.10 


DQ20 


0.73 


1.00 


4.00 


0.11 


DQ21 


048 


0.86 


5.00 


0.09 


DQ22 


1.02 


1.42 


5.00 


0.16 


DSUI^ 


8.19 


6.82 


34.00 


0.74 


DSUM22* 


8.67 


6.94 


34.00 


0.76 


SMABDF20 


13.69 


7.60 


38.00 


0.83 


SMABDF22** 


15.19 


8.36 


40.00 


0.91 



*DSUM22 represents the difference between the total test scores assigned by the two peer graders 
lor the same exam. 

**SMABDF2Z represents She difference between the sum of the absolute differences between the 
scores on all 22-ltems as assigned fay the two peer graders for the same exam. 
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The total test scores reported in Table 2 revealed a mucti closer agreement between peer 
graders than the actual absolute differences in scores they assigned for indivic&ai tesr items. The 
average difference between the two scores for the same exam was 8.67 poims (s.d. « 6.94). with the 
largest difference being ?4 points. As revealed in Table 4. when the absolute difference between 
score.' for each test item were analyzed, the mean for absolute cSfference was 15.19 points (s.d. « 
8.36). The largest absolute difference between the hwo sets of scores for the same examination was 
40 points. The mean foHhe absolute difference nearly doubled that of the mean for the recorded 
final test scores (15.19 to 8.67) for the same exam. To illustrate this point, one student who had a i- 
point difference between the total test scores assigned by the two peer graders had a 13-point 
absolute difference when the absolute difference points for the 22 individual items were sunned. 
Another student with a 7-point difference in the total test scores between the two graders was found 
to have a 29-point absolute difference across individual items. Even though there appeared to be a 
relatively high level of agreement arrong the peer gradei? for a identical test Item responses when 
total scores were confuted and then compared, in a majority of cases there were much larger 
differences between the scores assigned when absolute dfferences t>etween each Item rather than 
total exam score were contrasted 

In order to describe the extent of the relationship between the c£fferences t>etween the total test 
scores by the two graders and sums of the absolute differences for the same 22 items, a Pearson 
con-elation coefficient was generated. Across all 84 methods students, a coefficient of -.66 (p < 
.0001) was obtained. Coefficients of -.67, -.76, -.46. -.74. and -.83. respectively, were generated 
when scores for each of the five methods classes where examined separately. These negative 
coefficients indicate that the differences in total test scores by the two graders significantly 
misrepresent the extent of the actual differences that existed between the assessment of the 22 
individual items by the two graders. 

Tables 2-5 contain data that strongly suggest that, at least among these pre-sen^ice teachers, the 
two peers who graded each exam tended to hoki and use quite different expectations for what was a 
highly acceptable response for each Kern and that their expectations were often very different from 
those of the individuals who wrote the exam responses. For these pre-service teachers, what was 
necessary for acadenvc "su«»ss" for each item ai^ for the overall exam was fully in the "eye and rrrind 
of each grader.* This point is reinforced by the Infomnation received from each of the participants in 
one class who wrote pnor to the gracfing task that be or she would give him/herself "110" of "1 lO" 
points on the exam as he or she had answered it. The participants who earned •70," TS." and "78," 
respectively, on this exam were among those who claimed a peitect score prior to the peer grading 
exercise. 
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That these indh^duats would claim to have perfect papers is consistent with research literature 
desciit»ng how people tend to have confidence levels relative to the quaDty of their work that are 
frequently inconsistent with the actual quality of their ability, knowledge base, or performance 
(Qtenbeig, Sanocki, Epstein. & Morns. 1987: Moore, 1990; Weaver. 1990). These studies found Bttle 
agreement between setf-perceived levels of comprehension of specific sets of information and actual 
comprehension test scoies. In other words, the f eefings of knowing" (Koriat. Uchenstein. & 
Fischoff. 1980) or Illusions of knowing" (Epstein. Glertf)et9. & Bradley, 1984) that individuals have 
about what they know or have con^ehended are not very accurate or depertdable indicators of what 
they actually "know" or "comprehend" as measured by objective perfomnance tests. The findings In 
this study, although descriptive and not experimental, do lend support for the disparity cHed above 
behveen perceived accomplishment and actual accon^Iishmen: as revealed by objective test data. 

SECONDARY SCHOOL SOCIAL STUDIES STUDENTS 

Prepetiurea and Methotis 

The procedures applied during the pre-senfice social studies teachers study were identical to 
those followed by students in one secondary school class whose teacher volunteered to participate. 
The particular class was chosen because the dass was going to study the same chapter in the same 
book used in the methods students itudy. The identical exam was distributed atong with the same 
instnidions for the take home assignment. Multiple cof^es of the ranpleted exam were reproduced 
by the researchers for other steps of the study. The next day each student was 0ven an anonymous 
copy of the exam of two of his or her peers. The exams which had only code numbers on them were 
randomly cfisttibuted. No student graded his or her own paper or knew the names of the students 
whose paper he or she graded. As with the teacher exams, these students scored each item from "0 - 
5" and used their textbook to review any f nfonnalion that would help them decide what to accept and 
not accept for responses to each Hem. Of the 31 students assigned the set of tasks, 29 completed 
the tasks and turned in usable mateHals for the data analysis. 

Table 6 provkies the mean, standard deviation, and range of scores per each test item and for 
the overall test scores for both tests for these 29 students. These data also must be interpreted 
within She context that these students were given the take hoirte assigntront and told to complete it 
as correctly as possible since their responses would be graded. 



Tabie6at)out het. 
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Tabtee. 

Mean, jtardard deviation, and niinimum and maximum scores by the two peer graders for the 
individuat items and total test for the 29 secondary sociai stucfies students. 



VARIABi^ 


MEAN 


STANDARD 
DEVIATON 


NUNiMUM 


MAXIMUM 
SOQRE 


STD ERROR 
OF MEAN 


01 


4.71 


0.70 


2.00 


5.00 


0.09 


02 


4.29 


1.12 


0.00 


5.00 


0.15 


03 


4.29 


1.24 


0.00 


5.00 


0.16 


04 


4.45 


1.08 


0.00 


5.00 


0.14 


05 


4.34 


0.91 


2.00 


5.00 


0.12 


06 


4.47 


0.78 


2.00 


5.00 


0.10 


07 


4.38 


0.99 


0.00 


5.00 


0.13 


08 


4.03 


1.21 


0.00 


5.00 


0.16 


09 


4.34 


1.02 


0.00 


5.00 


0.13 


01 0 


4.12 


1.39 


0.00 


5.00 


0.18 


Oil 


4.21 


1.22 


0.00 


5.00 


0.16 


012 


4.64 


0.87 


0.00 


5.00 


0.11 


013 


4.40 


1.34 


0.00 


5.00 


0.18 


014 


4.14 


1.46 


0.00 


5.00 


0.19 


015 


4.67 


0.69 


3.00 


5.00 


0.09 


016 


4.67 


0.73 


2.00 


5.00 


0.10 


017 


3.78 


1.68 


0.00 


5.00 


0.22 


018 


4.55 


0.96 


1.00 


5.00 


0.13 


019 


4.52 


0.80 


2.00 


5.00 


0.11 


Q20 


4.29 


1.08 


1.00 


5.00 


0.14 


021 


4.55 


0.94 


0.00 


5.00 


0.12 


022 


2.83 


2.23 


o.oc 


5.00 


0.29 


SUIVI20 


87.29 


11.12 


60.00 


100.00 


1.46 


SUM22 


94.67 


12.27 


65.00 


110.00 


1.61 
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Whdn the differences beiween the scores given to the same exam by two p^ers were analyzed, 
many of the patterns found eariier In the methods students' data were found. The mean difference 
between the total scores for the same exam was found to be 10.45 points (s.d. - 8.47) with the 
smallest difference being one point and the laigest being 34 points. When absolute differences in 
scores were examined, the mean absolute difference between the two scores for aD 22-items was 
15.00 points (s.d. -> 9.19). with 39 points being the largest difference behveen scores assigned by 
the two peer graders, t-test analyses revealed the same pattern of significant difference for all 
(Variables exantined as reported above in Table 4 for the presenrice teacher data. 

These findings reveal that among studerus In this high school classroom, exactly what was 
necessary for an appropriate, complete response to these test ilen« varied widely from student to 
student. This difference could not be blamed on their having access to different infomiation they 
were to study or different exam Items to answer. The researchers concluded that a major contributing 
factor to these differences in test performance was that these high school students, like the 
preservice teachers, used their personally constmcted notiofK of what the cue tenns meant for them 
to do - and they proceeded to develop their test responses and assess their peers' answers on the 
basis of these notions. 

DEFINITIONS 
nontext and Procedures 

How Individuals define tenns s*rongly Influence how they Interpret, assign meaning to, and make 
decisions in situations where t'.iese temis arvi their meanings are important (Stahl. 1983). In this 
study, it was assumed that when these participants were asked, for instance, 'to identify the Battle of 
the Little Big Horn," their decisions regarding what Infonnation was a^^priate or inappropriate were 
based on their conceptions of what it meant "to identify" something. In addition, their definitions 
would also tend to serve as the criteifon for deciding the extent to which their own responses as well 
as those of their peers were correct, complete, ar^i acceptable. 

17)0se who completed the take-home exam and participated in the evaluation of the exams of 
two of their peers also defined 11 terms that often serve as cues in test items found In K-12 
classrooms. In the order of their appeararse on the iefinltlon assignment, these tenns were: identify, 
describe, explain, understand, comprehend, compare, contrast, define, analyze, discuss, and cause. 
The term "discuss" was the only terni not included as a cuing word in the items on the exam. This test 
and extended samples of the defiititions these preservice teachers wrote for these cue terms are 
provided in A{:^endix D. 
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This exercise provided infoimation about what these mr6$ meam to these pre-service teachers 
and high school students. Furthermore it enabled the researchers to collect information on the 
meanings these individuals had lor these cuing ternis and in« ustigate the extent to which the 
definitions for individual tern« were identical to or different from definitions of other temis on the same 
Ust. This procedure would also allow these participants to consider the differences among meanings 
they had tor these terms and what these differences might might mean for testing and assessment of 
students they were to teach beginning as early as the next semester during their student teaching 
experience. 

Findings Relative to The Meanings of The Cue Terms 

The 84 sets of definitions were examined to determine the extent to which they are identical, 
similar to or different from one ar»ther. (See Table 7 for a sample of these definiions for three terms.) 



Table 7 about here 



Results of the initial investigation showed a remarkable deg»> e of nonconfonnity in the 
definitions. For example, among the pre-service teachers, the tenn identify" was defined as "iocaUng 
information from a ^ven source in order to provide reinforcement for teaming." lo place or distinguish 
from others - to name." "to point out what is vital or Important." "to write down important infom>atlon 
about something but not as much as when you describe It." "to ^e a brief statement of who or what a 
person or event Is," and "to locate and describe an idea presented." The pattern of widely diversified 
meanings was consistent throughout the 84 sets of definitions. 

The definitions for the ten tenns c^en by the 29 secondary school students cfiffered as widely as 

those stated by the 84 pre-service teachers. This fitvilng revealed that the meanings for these cuing 

terms were as vari^ and Idiosyncratic aimng individual high school students a^i they were among 

these soon-to-be classroom teachers. 

This phenomenon of cSfferences anwng mear^ngs is consistent with the psychological 
findings that teamers at all ages and in every area invent and constnjct meanings to cues, tenns. and 
everytrur^ else in their wrorfcte (Stahl, 1983, in press: Wittrodt, 1986). Unless individuals are assisted 
in acxjuirir^ a particular definition, each wiR invent and use his or her personally icSosynuatic mear^ngs 
wirich will then viewed as being accurate. Such meanir^ may serve to be quite functionai In ty^^cal 
situations. However, In situations where one person's meanings are to be used to assess the 
perfonnance of another person, such as teacher-meaning used to assess student answers, it is 
important that these meanir^ be aOgned. Therefore teachers and students must spend time 
ensuring their meanings match in those areas where such meanings are to be used to guide student 
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Table?. 

San^ of definitions provided by the 84 preservice social studies teachers to the cue words on the 
definitions test. 

Identify 

~ pinpoint f»rson/plffi:e/thing and state what part they ptayed in what is being studied 

- noting spfhcHtc features 

- to pick out parts of a story or characters 

- to name or ^ate who someone is or what something is 
~ to categorize 

~ to categorize things 

- to know and exf^in 

~ to show that you know what it is 

- 8st the attributes of 

- express something or something 

- explaining the settir^ and stmcture and plot of what happened as completely as possible 

- to give details which tells abo jt someone, somethii^ or an event 

- explairting the details of items or kJees 
-Qive in detail 

- to explain In temns of appearances, characteristics of, etc. 

- to fist and explain specific details about a specific item 

- show understanding by exporting what it is 

Explain 

- using the attributes to show comprehension 

- discuss and write about a sftu^n 

~ discuss whatever happened as completely as possible 

- to ^e details, "reasons" to describe a concept 
~ discussing the meaning or reason 

-- cfiscuss the meaning of something 
~ to describe 

- to state in one's own words details about an everrt. object, or ad. 

- (fiscuas or write the mearting or reason for something 
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Study and resp^nsss and to evahiate student achievement. 

Tt»e area of most agreement of any of tlie cuing temts was for -cause." However, In reviewing the 
data concerrtir^ scores given for the item where participants were to 6st the cause of a patticular 
evb^it. toih the pre-service teachers and high school students were found to differ significantly in the 
scores they assigned for the same answers by their peers. Even though in their own minds these 
participants had somewhat simi'af meanings for what constituted a "cause." they did not agree on what 
specific information given in the chapter constituted a con-ect and complete answer relative to the 
cause of the event cited. Indeed, the differer^s among their actual exam answers regarding the 
cause{s) of the event cited were found to be stati^caSy significant (p <.00i). Thus, having somewhat 
sintiiar meanings for cue tenns is not, in and ?f itself, sufficient to guide students to locate and provide 
equally similar and acceptable histoifcal data congruent with the same cue tenns. Furthennore. if 
specific causes cannot be detennined given somewhat similar meanings and the identical historical 
data to study, then student success will rest almost entirely on the teacher's untaught criteria (i.e.. 
meanings). 

What JhPftP Differences In Meanings Meant to These Preservtee Teachers 

Not only did the pre-service teachers vary in their definitions of these 11 cuing terms, but there 
was also a wide disparity of opirrion concen^ng the implications that this variability might have on their 
assessment of students being the next serrester. Pre-service teachers in one secondary methods 
class were placed in small groups and a^ed to critiqued their test scores and explore reasons for the 
differences. After a lengthy discussion, nearty all agreed that the wide range of definitions and the 
failure to establish and then teach clear definitions to their students could only hinder their students' 
success. At the end of the group meeting, one wrote: 

-The implications of our differences pn test scores and definitions] are quite clear. 
Students would be confused, espedatiy if I didnl use the same definition myself from 
test to test or question to question... it seems to me that it would be unfair to 
students to use these temns. if teachers in the field already know that these tenns are 
unclear and thus are Bkeiy to have a negative affect on students, why haveni they 
decided to agree on definitions? They could at least do so in their own 
departmentsilir 

However, a number of presenrice teachers advocated a very different point-of-view. Typical of 
their responses is the following: "For the majority of definitions, we were in general concun-ence and 
in this regard we befieve this {(fifference] would not provide for confusion on the student's part. The 
ones IdefinitionsJ that were different began very similarly, but In some cases went off in different 
directions, though the basis of the definitions were similar. I.e., some parts of the definition were the 
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same * Among the definitions these participants thought mre in close agreement and wouM not tead 
to confusion in student thiniUng or responses were, for instance, for 'explain:' 1o tell how or why 
something woiks, or happened the way it does/did,* 1o place into perspective a particular object, 
thought, or j^ction." ai^ "the ability to transfer information from one person to another.* it was difficult 
for us to see these definitions as being suffidently simitar to one another so that an answer based 
upon any one would corresporKi well to answers generated by the other definitions. 

Writing on behalf of a group, one presentee teacher concluded that "they [the group] did not 
believe any great protjtems (for students] would result from these minor definitional differences." 
[Underline emphasis and bracketed infomiation added by the authors.] Quite remartcably, these 
teachc-s reached this conclusion after they had received less than perfect scores on thair exams by 
two peer graders and were reminded that each of them had expected a perfect score on the exam as 
she or he had oonpieted it. 

These reports were especially interesting in light of the fact that nearly alt of the pre-service 
teachers acknowledged in their groups that they themsehres tended to assign scores for individual 
answers based upon what they put down for their own answers or on what they thought the test item 
asked tor when they got set to grade the papers. Some reported they made their decisions about 
what was acceptable by comparing the answers on the two exams they were grading. In this latter 
case, they made a decision based not upon a single criterion but upon a comparison of the answers 
given on the two tests as they graded them. While they (fidnl think this was virtiat a teacher should do 
in aU cases, they considered it to be a lair way of evaluating answers when it wasm clear what a 
complete and con^ct answer should entail Yet. each of these preservtee teachers strongly rejected 
the notion that this procedure was "fair when their university professor suggested he might use 
these same non-defined cuing terms on exams and in evaluating these teachers' work in the methods 
course. 

CQNCLUStQMS IMPLICATIONS. ANH POSSIBLg COURSES OF ACTION 

tt is clear from these initial data that arvK^ng these pre-sen^ce social studies teachers, the terms 
used to cue responses to subject matter contem test items were defined differently among the 
individuals involved and that these differences generated dissimilar expectations as to what is a 
complete, acceptable response to items where these cue tenns are used.^ Furlhernwre. these data 
reveal t.iat the differences among total test scores are Bkeiy to misrepresent the actual differences that 
exist in the quality of the responses to individual test items when considered collectively in the form of 

^PieOminary data from one class of insen^ce social stucfies teachers revealed patterns of answers and 
grades very similar to those reported above for both preservice teachers and secondary school 
students. 
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absolute diff ereroe scores. 

The implicatior^ of these findings to (^te for student success in social studies classrooms are 
many. The discrepancy among the defiiiitions |30ses more than one prot)]em for students. Four of 
these are touched upon here. Rrst, if their nwaitins^ for the identical term are different from those of 
their teacher and if they use their personal definitions to form a response to a test item, the charts 
are great that students will not t)e given full credit on those items regardless of how much time and 
effort they put into framing and writlr^ this respor^. 

Secondly, if students are aware that their definition may not be the same as that of their teacher 
and if they desire to earn full crec£t for their responses to a particular item, then the students must 
spend time and effort during the test situation trying to invent meai^ngs of cue temns that will allow 
them to generate the responses the teacher expects. If their invented definition is correct, students 
will likely be successful for that test item response at that moment. If thPir invented definition is 
incon-ect. then their responses have little or no chance of being successful. In these Instances, the 
focus of student energy is w\ on teaming and revealing what they have learned or are learning. 
Rather, it is on survlvir^ In ^ualions where their success or failure as students depends on their 
ability to "read the nrtind" of their teachers for each test item on each test or assignrront. 

Thirdly, having somewhat similar meanings for terms may not be enough to guide individuals to 
locate or retrieve all of the particular information that the evaluator will expect as a totaUy acceptable 
response to a test item. In other words, being dose In meaning Is Bkely to produce different answers 
which in tum is very likely to result in (fifferent test scores- 

Fourthly, these fincSngs suggest teachers in a single department or school may find it valuable to 
develop clear and precise definitions to be used every time particular cue tenns are included in 
questions or directions. Time should then be taken to help students comprehend and apply these 
defirtitlons so that they become very skilled and cor»istent at using these meanings across situations. 
Students could then spend far more of their time and efforts in framing responses to meet clear 
expectations. Freed from the constant effort of "reading the mirKt" of the teacher as grader and 
evaluator. students can engage sooner and more fully in generating acceptable answers than they are 
able within ill-defined sftuations. Faculty members in touisviUe. KY schools where such definitions 
have been developed and used report teachers and students alike have found these to be very 
effective tools for all concemod. Many are very positive about having taken the time to generate 
these definitions as expectations and then using them in every course taught. Most importantly, they 
are positive about how it has helped students academically by eliminating a tot of the confusion, 
failure, and apathy that often came from ambiguous meanings. 

Even though much has been written about constnjcting test Items and assessing students in 
social studes classrooms (e.g., Airasian, 1991 ; Brown. 1976; Moore, 1980; Williams & Moore. 1980). 
there is little evidence that these guideBnes and notions are widely used by classroom teachers or 
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significam mimbers of cotnmsfcial pubMshers. While the Hems on the exam used In this study were 
cited as t>eing "poor by several critics at a recent national conference of the Nittional Council for the 
Social Studies, the wording was tiftad from samples of test Hems and adivHies written tsf and collected 
from a number of elementary and secondary classroom teachers, district test designers, and 
commercial materials witWn the past five years. The wordir^? was typical of what we have found in 
analyzing many tests used on different levels of social studies instruction across a number of states. 
While all teachers do not write test items using such tenns without providing clear definitions of what 
they require, these terms appear often enough in questions used for discussion, tests, and 
worf<sheets to warrant examination by teachers of the terms they use to cue students for teanting 
success. Most in^ortantty, we are not suggesting that these term be eKminated as ojing ternis for 
student responses: we are advocating that for whatever temis are used, teachers and students have 
agreed-upon meanings that translate into clear expectations of what is needed in the response to 
receive full credit. 

To date, a maiority of edi^ators at an levels and aooss all contem areas have retisted the effort to 
devetop a mutually agreed upon precise definitions for the cue temis they frequently use to guide, 
measure, and evaluate student leaning. At the same time there Is a major concern over what leads 
students to be "unsuccessful" and "unnwtlvated" In pre-college classrooms. As Indicated in this 
study, the differences in definHions arKl meanings teachers and students have for the identical cue 
terms contribute significantly to the extent of student success or failure. 

Considering the above, the chances are great that large nun*ers of students may not be doing 
as well as they could be in social studies classes because of the ambiguity of the meanings of the 
tenns that are used to focus their learning, guide their responses, and assess the quality of those 
responses. While developing and using such definitions will not remedy every problem in these 
classrooms, this Is one positive step every teacher can take to eBminate problems that result from this 
ambiguHy. This notion is supported by the work of Bloom (1976). Guskey (1985). and others who 
report that students, including less able, slow, and even seemingly unmotivated ones, tend to t>e far 
more successful when they are dear as to precisely what they are to leam, how they are to learn It, and 
how they are to demonstrate what they have achieved. 

Educators can take steps to examine the differences among the definitions that so greatly 
influence achievement and begin development of unHomt parameters for each commonly used cuing 
temt In this way, test items csan f ureaion to find out VhaT students actually have teamed relative to a 
clear set of expectations rather than revealing the resutts of students' efforts to "sunnve" in the 
arbitrary grading jungle of the social studies classroom. 
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Examination ^ 

You arc to use only the materials provided in the text paces vou were given to 
answer the Questions below: 

1. Who was Chief Joseph? 

2. What was the "Great American Desert"? 

3. Describe the "Great Plains Indians." ^ 

4. What was the "new Indian policy"? 

5. Identify General Gecrge Custer. 

6. Describe the Battle of the Little Big Horn? 

7. Identify: The Battle of Wounded Knee 

8. Analyze the destruction of the buffalo. 

9. Explain ihe advance of the railroads across the Great Plains. 

10. What caused the "new Indian policy"? 

11. Compare and contrast the Plains Indians and the white man in the Plains 
area from the lS30s until the end of the Indian wars. 



ERIC 



BEST m ^VM^^^^ 



ERIC 



Achieving a High Test Score in the -^ii! StudtesCiassroom: ^spendicds 26 

12. Define: reservation 

13. Identify: concentration 



14. When did organized resistance against the white man end in the West? 



15. Describe: Chief Sitting Bull. 



16. Who was Chief Crazy Horse? 



17. Analyze the movement of the settlers into the Plains region just before and 
soon after the Civil War. 



18. Explain the events leading up to the Battle of Wounded Knee. 



19. What the the most important theme or idea that was emphasized in this 
chapter? 
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20. Why is your answer lo number 19 the most imponani ihcmc or idea in this 
chapiei? 



21. It is expected that after you firishcd reading and studying this chapter that 
you understand what the chapter was all about. In ihc space bclov.', write one or 
more paragraphs thai will let me know tl.at you understand the information in 
the chapter about this era in the history of the Great Plain.s; region.. 



22. Use the space below to let me know that you comprehend the information in 
the chapter about this era in the history of the Great Plai is region. 
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APPENDIX B 



SAMPLES OF RESPONSES TO SELECTED ITEMS ON THE 22-ITEM 

EXAMINATION 
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Table B1. 

A sample of the responses ot social studies methods studems to and the scores of the two peer 
graders on an Item on the 22-item take-home examinalion. 

iTgM; Anfllvge ihP movement of the settlers Into the Great Plains reolon tust 
before and soon after the Civil War. 

a) They first moved through the Plains on their way to the Pacific and described it as a great desert. 
But then began to realize they coufd fanri and raise cattle there. (3)(5) 

tj) Pnor 10 the Civil War (1848) settlers were very hesitant to move into or onto the Great Plains. Lite 
was hard, lack of water. Indians, lawlessness, little game. As time went on settlers discovered 
how good and fertile the land was for fanrtng and grazing. Yet the Incfians were still prevalent. 
The government set up policies to allow more settlers to move in. Keeping the Indians 
concentrated. After the war, many soldiers from both sides needed work, so fighting Indians 
becanw an easy end to their means. Thus more and nwre moved in as the Indians became fewer 
and fewer. (4)(5) 

c) Very sporadic, first on horseback, wigons, and then stagecoaches. After the Civil War, Great 
Plains attracted Increasing numbers of land speculators, ranchers . miners, engineers, and 
farmers. These newcomers were determined to possess the land and its resources for 
themselves. (3)t4) 

d) The movement of settlers into the Plains region was slow before the Civil War. They were 
threatened by harsh conditions. Indian attacks, and forms of stow transportation. As the 
stagecoach began to be used, more of the railroads built, the settlers were able to penetrate the 
temtory more easily. (3}(3) 

g) Prior to the Civil War most settlers befieved the Plains to be uninhabitable. By the 1850s they 
realized their mistake as reports of good farm lands and cattle raising areas became apparent. 
The development of the railroads and the end of the Civil War were also factors in this expansion. 
(5){4) 

h) Before the Civil War, settlers were just beginning to move Into the Ptalons region, recieving 
reports that the region actually was good for farming and cattle raising. After the Civil War, 
increasing number of settlers moved into the area, not only farmers and ranchers, but also land 
speculators, miners, and engineers. (4)(5) 
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Table B2. 

A sample of the responses of social studies metfiods students to and \t\e scores of the two peer 
graders on an item on ttie 22-item take-home examination. 



ITgM! Deserl bi* the Battte of the LHtle Bio Horn? 



SO General Custer and 264 troops attempted to remove Cheyenne and Sioux Indians from that area. 
Sitting Bull and Crazy Horse were the Indian leaders and they kilted Custer arti atl his troops. 
(4){5) 

b) General Custer and 264 troops attacked a large party of Sioux and Cheyenne Indians led by 
Crazy Horse and Slttir^ Butt. The battle occurred i^ar the Little Big Horn River in Montana. All of 
Custer's men were kiUed. marked the end of ma^r fighting in the noithem Great Plains. (5}(5) 

c) In June. 1976, Custer attacked a large camp of Sioux and Cheyenne near the Uttte Big Horn 
River in Montana. Sitting Butt and Crazy Horse defeated him and his 264 troops. Major 
humiRation. (4)(4) 

d) The Battle took place in Montana. It consisted of the Sioux and Cheyenne headed by Sitting Bull 
and Crazy Horse who were fighting their removal to the reservation. They were successful in 
defeating the calvary, but ultimately settled on the resen^ion. (4){5} 

e) An attack lead in June of 1876 by General Custer against a large group of Sioux and Cheyenne 
Incfians camped near the Uttle Big Horn River in Montana. The Indians were lead by two 
outstanding leaders, Sitting Bull and Crazy Horse. This became "Custer's las\ Stand" as he and 
his who detachment of 264 troops were killed. (4)(5} 

f) Custer attacked village. Indians led by Crazy Horse and Sitting Bull killed ail Custer's men and 
mutilated bodies of calvary soldiers. (5}(5) 

g) Battle of the Uttle Big Horn was a fierce battle in 1876 in wtiich General Custer attacked Sioux 
and Cheyenne Intfians along the Uttle Big Horn River in Montana and was defeated. (4)(4) 

h) in this battle. General Custer and his troops attacked a large camp of Sioux and Cheyenne 
Indians In Montana and were all killed. The battle was a hunv'liation at the time for the 
govememtn, but brought a tenporary end to fighting in the area as the Indian leaders from the 
battle were soon pursued and captured. {4)(5) 
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Table B3. 

A sanpte of the responses of sodaf studies methods students to and the scores of the two peer 
gracters on an Hem on the 22-item take-home examination. 



ITgM- ComPffro and eontrart the Pialiw i Indians and thP white man In thg Plains area 
fr«m the isans until the and of the Indian wars. 



a) The Plains Indians roamed the plains hunting mainy buffalo. As they were forced onto 
reservations they resisted with great intensity. The white man on the other handsaw the plains as 
a place for white people to own land and five on. They felt the Indians must be concemraied on 
reservations. The indans saw no ownership of land, no boundaries, the white man did. (3)(2} 

b) The white man of the 1830s stayed to the east of the 100th meridian generaUy. The Plains 
imfians (hiring this period were "veterans" of the land. They knew how to farni and hunt the "arid" 
plains. By the 1850s the notion that the plains were desolate wastelands was disposed and more 
and more settlers kept moving westward. The Indians had an "edge" early In the century with 
their weapons . but as the revolver was invented and became more available, settlers were more 
compelled to move to the plains. As the Incfians' resistance weakened, white men increased on 
the scene. The Indians were conserving, environmentally conscious people (buffalo), whereas 
the whites were wasteful greedy people. The Indians continued to lose land and the whites 
continued to gain land. Both fanned, grazed, and hunted on the land. C3)(5) 

c) m the 1B30S. the Ptelns Indians enjoyed relative prosperity and trant^llity in the Plains area 
because the white man felt the land was worthless. However once that misconception was 
reafized. the white man warned their piece of the land, regardless of Indian considerations. This 
led to strife which lead to the Indian wars and eventually ended in the bnital massacre of Indians 
at Wounded Knee. (2)(4) 

g) The white man and Indians seemed to trade superiority during this time. Both with the common 
goal of control of the land. In the 1830s the Indians were superior to the white man in n^any ways: 
adaptablBty to their environment, burning, riding, as warriors. By the late 1830s and the inveniion 
of the revolver, white men steadily Increased in power, breaking promises and forcing Indians 
onto resen/ations. {3)(4) 

h) Early In this period, the Plains Indians recognized the abundance of the Plains area for 
subsistence, whereas the white man saw the area as an arid, ur^nhabitable desert. By the 
1850s, the white man too recognized the value of the Plains for making a living, but still 
approached the task from a different perspecth^ than the Indians. The Indians made their living 
off the existing resources of the Plains, the white man attepted to remove the main animal 
resource, the buffalo, and replace it with their own cattle, as welt as Introduce agricuHure. (4)(5) 
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A sample of the responses of social studies methods students tc and the scores of the two peer 
grades on an item on the 22 -item tane-home examination 



tTEM: When did organiz ed reststance aoatnst thp white man end In the West? 

a) The Battle of Wounded Knee. <4)(5) 

b) In 1B86 organized resistance In the Southwest and Southern Plains came to an end with the 
sun-enders of Geronimo In 1886. In the "wesr in genera!, the Battle ot Wounded Knee. (4)(4) 

c) Wrth the surrender of Geronimo in 1886. (5K5) 

e) After the Battle of Wounded Knee endea in South DaKota m 1S90. (0)(i) 
t) After the Battle of Wounded Knee (1889) althoush t)attle' doesnt sound Mke the Indians were 
either "organized" or actively resisting. (5)(5) 

g) Organized resistance against the white man came to an end in 1886 after three centuries of 
warfare when Geronimo surrendered. (5)(4} 

h) Organized resistance against the white man ended in 1886. when the Apache Chief Geronimo 
surrendered. {4)(5) 
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Table B5. 

A san^e of the responses of social stucSes methods students to and the scores of the two peer 
graders on an item on the 22-item take-home exanvnation. 

iTgM: Who was Chief JosBPh? 

a) He was the tesder of the fslez Perce Indians. (5){3) 

b) Leader of the Nez Perce Indians. Led his people away from their homeland. Surrendered to the 
wnlte man. (5){5) 

c) Leader of the Nez Perce Indians whose tribe, after trying to run to the Canadian border, was 
surrounded by American troops who wanted to relocate them on a reservation outside the fertile 
Great Plains. (4}{5} 

d) Chief Joseph as the leader of the Nez Pierce Indians, a Pacific Northwest tribe. (3}{3) 

e) Leader of a group of Indians known as the "Nez Perce." They were a peaceful tribe who had 
Oved where the present day states of Oregon, Washington, and Idaho meet. Chief Joseph 
surrendered to the U.S. Army in 1877 in Montana near the Canadian border. His people were 
freezing to death. Chief Joseph had been attempting to lead his people to a free land in Canada. 
(5){5) 

f) Leader of the Nez Perce Indiats who choose to run rather than fight the U.S. caverty. (5)(3} 

g) Chief Joseph was the leader of the Nez Perce Indians who had Bved on the Great Plains until 
pioneers invaded the land, ^le became famous for his attempt to flee to Canada and ultimately 
surrendered to U.S. troops very close to the Car^cSan border. (5)(4) 

h) Chief Joseph was the leader of the Nez Perce Indians in the 1870s. He attempted to lead his 
people to a new homeland in Canada when the United States govemment ordered them out of 
their original lands. (5)(3) 
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Table B6. 

A sample of the responses ot social studies methods students to and the scores of the two peer 
graders on an item on the 22-item take-hon« examination. 

ITEM: What was the "Great American Desert?" 

a) The plains. (3)(1) 

b) The area of the Great Plains. The white settlers were accustomed to trees and abundant water so 
when they got to the grassy plains it appeared liKe a desert to them. {4)(5} 

(c) The Great Rains. Perceived by the Federal Govt is being arid and uninhabitable . However was a 
misconception and traders and "pioneers* wanted to settle there. (3}(3} 

d) The "Great Anierican Desert" was that area west ot the 1 00th meridian ano east of the Rockies. It 
consisted of states such as the Dakotas. Nebraska, Wyoming, Kansas, and Oklahoma. (3)(4} 

e) Also known as the "Great Plain," It reached east to west from the 100th meridian toward the 
Rocky mountains. A unifying factor of the area Is the extremes of climate, including rain stonDs. 
tornadoes, hail, and wind. (4)(4) 

f) Name given to the great plains in the early part of the I9th century. Name was based on mistaken 
belief that the great plains were arid and uninhabitable. (5)(5) 

g) The Great American Desert was land designated by the lOOth meridian (now known as the Great 
Plains) which was named during the earty 1800s by explorers who considered the areas 
uninhabitable. (4}(4) 

h) The Great American Desert was th*? name explorers and eariy settlers gave to the Great Plains in 
the first half of the 1800s. Since the region was not wooded it was given the title "desert," a dry, 
uninhabttat)le area. (4)(4} 
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Table B7. 

A sample of the responses of sociat stucSes methods students to and the scores of the two peer 
graders on an item on the 224iem take-home examination. 

ITEM: Descri be th» "Greet Plains Indians." 

a} They hunted buffalo and were expert horse riders. They were also very good warriors. (3){4) 

b) They mainly consisted of previously 'relocated' Eastern Indians that the government and whites 
in general "pushed" to an undesirable environment. These Indians were great hunters and 
fanners, and knew how to "break" wild horses and fight off Intruders until the invention of the 
revolver. (3)(5} 

c) Determined to defend their huntif^ grounds and their way of Ufe. Were expert riders. Hunted 
buffak) which provided them with food, ctothirtg, and shelter. Powerful adversaries. Rode 
expertly. Fought with ^ars first before the rifle. (5)(5) 

d) The Great Plains ImSans were groups of r^imadic {copies that roamed the plains region on 
horses. They followed their food source, the buffalo, from which they made much of their 
supplies. (5}{4) 

e) These were the tribes of Arapaho. Crow, Comanche, Mandan, Sioux. Ankara. Cheyenne, and 
Pawnee Indians who lived in the Great Plains area. They were highly skilled hunters who fought 
with arrows and long spears on horseback. But with the advent of revolvers, they were no longer 
a match for the wh(*3 man. (4)(4) 

f} Various tribes, often at war with each other, primarily nomadic followers of buffalo herds, 
characterized t>y their horsemanship and use of bow and arrow. (5)(5) 

g) The Great Plains Intfians can be described as expert riders, excellent hunters and and powerful 
adversaries who adapted themselves to the Plains and remained superior to white settlers until 
!he1B30s. (5)(5) 

h) The Great Plains Indians were tribes that had adapted to the plains environment for their Oving. 
They hunted on horseback the abundant plains buffalo as a source of food, clothing, and 
shelter. (4}{5) 
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Table B8. 

A sample of the responses of social studies methods students to and the scores of the two peer 
graders on an item on the 22-ttein take-hotne exanvnaSon. 

iTgM! What was the ^'new Indian DOtlcv?" 

a) It was a |:»Bc^ of "corK^ntratlon," where the new Bureau of Indian Affairs confined Indian tribes to 
fiiT^ed areas. (2}{4) 

b} A poficy develoi:»d by the BIA to concentrate the tribes into certain areas (i.e., reservations) so 
as to fimit where the Indians could live and give whites traveling to the far west easier access. 

(4) (5) 

c) 1849 - Bureau of Indian Affairs became part of the Department of the imenor and Its policy was 
concentraSon in an attempt to confini; the Indian tribes to certain Bmited areas of the West where 
they could five their own fives and hoped tt wouM reduce warfare between tribes and clear them 
in areas for whites headbig for Cafifomi& (4){^ 

d) The new Indian poTrcy was one of "roncentratloa" The Department of intenor wanted to gather 
the incfians into a group so as to make more room for the white settlers. (4)(4) 

e) in 1849 the Bureau of Indian Affairs became part of the Dept. of the interior, and new policies 
toward the imfians emerged. One policy was "concentration" which hoped to assign tribes to a 
certain Bmited area so that f >^<* white settlers could have clear, peaceful routes to CaFifomia and 
Oregon. (5){5) 

f) Was an attempt to conHne Indians, by force of amis if necessary, to reservations and to thereby 
convert them from nomadic hunters to fixed plat fanners and/or dependents of U.S. Federal 
programs administrated by "incSan agents." (4}(5) 

g) The "New Indian PoDcy" was an attempt by the Bureau of Indian Affairs to confine Indians to 
Cmited areas in the hopes of reducing tribal warfare and :iear routes for pioneers heading west. 

(5) (4) 

h) The "New Indian policy" was to greatly Unlit the range of Ir^an tribes on the plains and so opened 
much new land for settlement by whites and reduce the danger of Indian attacks on overland 
travelers. (4){4) 
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Table B9. 

A sanple of the responses of soctai studies methods students to and the scores of the two peer 
graders on an item on the 22-item take-honie examination. 

ITgM! identify General Georoe Custer. 

^ Custer was an Army General who led troops against the Sioux and Cheyenne and was killed with 
all his men m the Battle of the UtUe Big Horn. (5){5) 

b) An exi^rienced General and trofian f^hter. He had a history of brutal massacres credited to him. 
Was the general who was defeated and killed at the battle of the UtUe Big Horn. (5}(5} 

c) An experienced Indian fighter, leader of the 7th cavaty. Defeated at Little Big Horn. Several 
years earlier attacked a peaceful Cheyenne incfian village and lulled unarmed women and 
children. (4)(5) 

d) Geoige Custer was the General of the 7th Cavalry. He is best known for his defeat by the Sioux 
at Uttle Big Horn. (5){5) 

e) A General in the Amiy who commanded the 7th Cavalry. He was an experienced Indian fighter. 
{3)(4) 

f) Experienced Indian fighter who had massacred a peaceful Indian village on the Washita river. 
General slaughtered after attacking an Indian viUage on Big Horn River. (5)(5) 

g) Custer was an "experienced Indian fighter and leader of the failed U.S. Cavalry removal 
operation of the Sioux tncfians out of the Black Hlls. {5}(4} 

h) General George Custer was an experienced, sometimes ruthless, Indian fighter who became 
famous for a battle he ted his troops into against the Sioux and Cheyenne Indians. He and ail of 
Ns troops were killed in the battle. (4)(5} 
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Table BIO. 

A sample of the responses of sociai studies methods students to and the scores of the two peer 
graders on an Hem on the 22 -item take-home examination. 

ITFM- nAsrrlhP' Chief Slttlno Bull. 

a) The leader of the Chevenrm Indians he was described as honest, able, and ideaUstic (1 )(5) 

b) Able, honest, and idealistic Sioux leader and chief. Led the Sioux over Custer In the Battle of the 
Uttle Big Horn. (5)(5) 

c) able, honest, idealistic, i assume hes Cheyenne. tHJt not sure. (4){3) 

d) Chief Sitting Bull was the leader of the Cheyenne warriors. He was able, honeSi and idealistic. 
(5)(4) 

e) He was the able and honest and ideafistic leader of the Cheyerne warriors who fought Custer at 
the Battle of the Uttie Bg Horn in Montana in 1876. (5)(3) 

f) Able, honest, idealistic. (5)(4) 

g) Has been described as an outstanding warrior considered to be honest and idealistic who aided 
in the defeat of Custer at Uttle Big Horn but who eventually was defeated by the U.S. Cavalry 
troops. (5){5) 

h) Sitting Bull was an able, honest, and ideaUstic leader of the Sioux Indians, one of the two leaders 
who played a major rote in the defeat of General Custer at the Little Big Horn. ( ;>}(5) 
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Table B11. 

A sample ol the responses of sociat studies methods students to and the scores of the two peer 
graders on an item on the 22-item take-home examination. 

ITgM; Whf was Chief Crazv Horse? 

^ The leader of the Sioux Indians, Crazy Horse was describe as uncompro nising, recl<iess and a 
milita7 genius. (5)(4) 

b) Chief and leader of the Sioux. An uncompromising, reckless military genius. Most honored hero 
of the Sioux. (5)(5) 

0) Uncompromising, reckless, a military genius and most honored hero of the Sioux. (4}(5) 
d) Crazy Horse was the leader ol the Sioux tribe and he was uncompromising, reckless, a military 
genius, and the most honored hero of the Sioux. (5)(4) 

f) uncompromising, reckless, honored hero amor^ Sioux, and a military genius. (5)(4) 

g) A Sioux warrior known as their most honored hero, considered to be a military genus but 
extremely reckless. He also figured prominently in Custer's last stand but then was defeated and 
forcedbythe U.S.toBveonaresen^on. (5)(5) 

g) CNef Crazy Horse was a leader of the Sioux, their most honored hero, and one of the two leaders 
who pteyed a major role in the defeat of General Custer at the Little Big Horn. (3){5) 
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Table B12. 

A samfAe of the responsvs of social studies methods students to and the scores of the two peer 
graders on an item on the 22-item take-home examination. 

iTgM- Identify- TM» BaMie of Wounded Knee. 

In 1870 a group of Sioux Indians were traveling In search of food. The Anriy Intercepted them 
and cfisarmed them. A shot rang out and the troops opened fire on the unamied Indians. 290 
men. women, and children were killed. (4)(5) 

b) Evert that ended all organized armed resistance by Indians In the Great Plains. An "innocent" 
band of Sioux were halted or detained during a travel in which they were searching for 
food/protection. One shot caused an outbreak, kilting many. Resistance caused by the Ghost 
Dsncfi sc3r6» 

c) Should be ttie massacre at Wounded Knee. Where troops murdering 40 men and 200 women 
and children at Wounded Knee. South Dakota. Stopped organized resistance against tncfians - 
1890. (4}(4) 

d) The Battle of Wounded Knee was a cortfrontation between the 7th Calvery and the Sioux who 
were on their way to Pine Ridge Reservation. Because resent fears about Indian resurgence 
they were surrounded by soicfiers and killed when a distuit}arK:e broke out. (4}(3) 

e) White settlers fearful of the "Ghost Dance" movement among the Indians, which called for an 
Indian messlah to appear, demanded that the Army put an end to this Indian activity. At 
Wounded Knee in South Dakota in December 1890. a unit of the 7th Cavalry an^ested a band of 
Sioux IncSans bound for the Pine Ridge Reservation. A disturbance broke out and the troops 
usir^ gatling guns klHed alt the disamied Sioux (90 men and 200 women and children.) This 
action ended all organized resistance from the Indians. (S)(5) 

f) "Battle" in 1889 where 7th cavalry opened fire on group of unamned Sioux men, women and 
children. (4)(4} 

g) Battle of Wounded Knee was a massacre by U.S. Cavalry in South Dakota in 1890 of Sioux 
Indians which Wiled almost 300 - mostly women arKf children. (4)(4) 

h) The battle of Wounded Knee was a massacre of 290 unanned Sioux Indians by government 
troops. This group of Indians was travefing peacefully to a reservation, but was apprehended by 
the troops in response to the fears of some settlers that an Indian uprising was imminent. {4)(5) 
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Table Bi3. 

A sample of the responses o1 social studies methods students to and the scores of the two peer 
graders on an item on the 22-item take-home exanvnation. 

ITEM: Analvzg the destruction of thg buftala. 

a) The expansion of the railroads divided the buffalo herds, but the main reason tor the destruction 
was a poBcy of the government to reduce buffato herds to keep the Indians under control. This 
)K>licy encouraged the slaughter of the buftato. (4)(5} 

b) According to this text's version of the destnjction. the buffalo were killed in order to "doom" the 
Indians, knowing that they were utterly dependent on them. For various reasons, the 
government condoned such actions as a means to "choke" the Indians. {3}(5} 

c) Plains Indians depended upon them with blessing from Federal Govmt who wanted to confine 
them to resen^ations. Hunting parties killed them and used their hides for eastern markets. 
(3){5) 

d) The destruction of the buffato took place In the 1870s. The purposeful anulDhiiation of the herds 
was designed to keep the Indians on its resen/ations by depleting them of their source of goods 
and food. (5)(4) 

f) As rail lines began to cut across the plains, migratory routes of buffalo were destroyed. 
Additionally, railroad brought whites who wantonly destroyed herd of buffato. in part because 
they believed herd destruction would keep Irefians on resen^tions. (5K3) 

g) The destruction of the buffalo was a delberate inten/ention and encouragement by the U.S. 
Govt of hunters to destroy buffato and eventually forced Indians out of control of the Plains. 
{5)(5) 

h) The buffalo were destroyed by white Americans both directly (from "hunting" parties) and 
indirectly (from the railroads cutting across buffato migratory routes). Destruction of the buffalo 
was encouraged by th; government, as f^uch action would take away the livelihood of the 
Indians. (4)(5) 
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TaWe B14. 

A sample of the responses of social studies methods students to and the scores of the two peer 
graders on an item on the 22-item take-home exanunation. 

ITEM; gxpiain the event s teadlniy up io the Battle of Wounded Knee. 

^ t: e Indian tribes of North America were caught up In a refi^ous revival tjelleving that an Indian 
Messiah was to appear and free them from the whHe intruders and the dead would rise. The 
wf^e man called iWs the "Ghost Dance " The whites also saw this revival as a movement toward 
war again. AnrM pressure from many ii^pfe the Army tried to put a stop to this movement and a 
caivaty unit responded at Wounded Knee. (4)(5) 

b) In 1889 the Intfian tribes of the Northern Plains celebrated a 'revivar called a "Ghost Dance.* 
The dance wasnt a caU to war, only fa reB^ous revival based on the t^tief that an Indian messiah 
was about to appear. With it came widespread popularity from other despairing tribes in the area: 
the dance quickly gained many followers. White settlers and miners panicked at the idea of an 
outbreak of war, and demanded the anny be sent to quell the "activfties." The 7th Cavalry, in 
accordarKie, came on the seen and quickly went to wortc In 1890 In South Dakota, they anested 
a band of Sioux men. women, and children and then the "battle" began. A dispute, then total 
if^anity broke out. (4)(5) 

c} In response to the "Glu)^ Dance" cult, a rsUgious revival movement, which featured the coming 
of an Indian messiah who should return the buffalo and expel the white man, white miners and 
settlers, alamned at what iney feared might be another outbreak of warfare, demanded that the 
amiyputanendtothteKti^. (3)(5) 

d) The Ghost Dance was a significant event that led upto the Battle of Wounded Knee- The Ghost 
Dance was part of a refigious revival spurred by a prophet that had a vision of an Indian messiah 
about to appear to free the Indian people. Miners and settlers got wind of this and became afraid 
of a new Indian resistance movement. They demanded that the army keep a watchful eye on the 
IncSans for signs of any trouble. Tensions were high and as a band of Sioux approached the Pine 
Ridge Resenration for food and shelter, they were shot down. (290 people). (5}(3) 

g) The Battle of Wounded Knee was predpftated by a refigious revival by the Sioux called the Ghost 
Dance. This revival was thought to be a call to war by the white settlers arvi eventually caused the 
settlers to demand that the cavalry step in and put a stop to It. (3)(5) 

h) In South Dakota, as In other porttons ofthe Plains, the Indians had been rek>cated to resen^ations 
by the late 1880s. and white settlers and miners had moved into much of the sumounding land. 
In 1889. a reHgious revival, the Ghost Dance, annong the Indlsnds brought many whites to fear 
that an Indian uporising was Imminent. The whites therefore denranded that the government 
send athe army to the Plains area to stop the dangerous reBgious activity. The army cam 
prepared to control the Indians both on and off the reservation. The evems can be explained by 
each group's unease at Its position relative to the other group. The Indians wished to have their 
spactuos ttradilional lands back free of white interference - the central idea in their religius 
revh^al- while the whites didnl feel free from the danger from the Indiands who had recently 
occuppied the area and had fought savagely to keep it. (5){5) 
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Table B15. 

A sarnple of the responses of social studies methods students to and the scores of the two peer 
graders on an item on the 22-item t^e-home exan^nation. 

ITEM: What was thg most Important theme or Idea that was emPhBStzed In this 

a} The theme is that the white man broke promises ma6e to the Indians in their (the white man's) 
quest to settle the plains. {5)(5} 

b) That the Indians were constantly undermined by the repetitive advancements by the white man. 
The Indian tosing his position and status. (4)(4) 

c) That the American Indian War constantly victimized the Indians In this period by expanslomstic 
Federal-white man goate. (4)(5} 

d) The most important theme is that while the settling oi the frontier by the whites was prosperous 
ai%l helped to build a great nation, ft extraded a heavy toll on the Native American Indians. (4)(5) 

e) The opening of the Great Plains to the American white settlers, at the expense of the Indian 
dvifization. (5)(5) 

f) That the Indians were continually made promises that were then broken. (4)(3) 

g) The most Important theme seemed to be the white man's unfair and often brutal treatment of the 
Indians which occunedwrhilesettlir^ the Plains. (4)(3) 

h) The nrosl important theme was how, and why, the indaians were making their last stand on the 
Great Plains. (4}(4} 
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Tabte B16. 

A sample of the responses of social studies methods students to and the scores ot the two peer 
graders on an item on the 224iem take-home examination. 

ITEM! Why Is your answer to number 19 above the most Important theme cr Idea In 

trtis ctiapter? 

a) It Is the most important theme because It describes the meanings of why the white man and the 
Indian reacted the way they did to the events. (5)(5) 

b) Be(^se It occurs or shows up in alnK)St every section and paragraph In the chapter. It is an on- 
going theme, with events to back it up continuously. (5)(5) 

c) Because it must be acknowledged that the plight of the American Indians at the hands of the 
Federal Govt dkl happen. (4)(5) 

d) it shows that whUe H helped build our country it also helped to destroy a race of people and its 
culture and tracBtions. (4)(5) 

e) The focus ot this chapter was not only to teD the "History" of this period, but also describe factors 
which contributed to the theme (e.g.. the buffato. the railroad, the settlers). These events and 
factors converged to tell the "Story" which emerged as the theme. (3)(5} 

f) Because It Is mentioned the most often and It underfies the description of various events. (5}(5) 

g) Throughout every subunit or Idea expressed in the chapter, the undercurrents emphasize this 
treatment. (4)(4) 

h) All of the other ideas tie into this theme at some pom and most of the narraTive deals directly with 
that main theme. (4)(4) 
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Table C1. 

Mean, standard deviation, and minimum and maximum scores by the two peer graders for Individual 
Hems and total test tor the 1 1 preservice social studies teachers in Group A. 



VARIABLE tJEAN STANDARD M04IMUM MAXIMUM STD ERROR 







DEVIATION 


SCORE 


SOOF^ 


OF MEAN 


w 1 


4 18 


1 10 

1 * 1 V 


2 00 


5 00 


0.23 




3 77 


0 92 


1 00 


5.00 


0.20 






0 78 


3 00 


5 00 


0.17 




A 1R 
•t. 1 o 


0 Q1 


2 00 


5 00 


0.19 


0*5 




0 67 


3 DO 


» 5.00 


0.14 


OB 


4 50 


0 51 


4 00 


5.00 


0.11 


07 
w/ 


4 23 


0 61 


3 00 


5.00 


0.13 


Oft 


3 86 


0 83 


3 00 


5.00 


0.18 


OO 


4 14 


0 99 


1 00 


5 00 


0.21 


O10 


4 45 


0 67 


3 00 


5.00 


0.14 


011 


3 64 


1 18 


2 00 


5.00 


0.25 


01? 


4 00 


1 23 


1 DO 


5.00 


0.26 


Q13 


4.00 


1.51 


0.00 


5.00 


0.32 


014 


3.82 


1.44 


0.00 


5.00 


0.31 


Q15 


4.14 


1.04 


1.00 


5.00 


0.22 


Q16 


4.64 


0.58 


3.00 


5.00 


0.12 


017 


4.05 


0.79 


3.00 


5.00 


0.17 


018 


4.27 


0.83 


3.00 


5.00 


0.18 


019 


4.09 


0.68 


3.00 


5.00 


0.15 


O20 


4.32 


0.72 


3.00 


5.00 


0.15 


021 


4.05 


0.79 


3.00 


5.00 


0.17 


022 


3.73 


1.28 


0.00 


5.00 


0.27 


SUM20 


83.14 


9.22 


62.00 


97.00 


1.97 


SUM22 


90.91 


9.29 


70.00 


106.00 


1.98 


GRADEGPA 32.73 


6.27 


20.00 


43.00 


1.34 
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Table C2. 

Mean, starxlard deviation, and minimum and maximum scores by the two peer graders for individual 
items and total test for the 25 presennce social studies teachers in Group B. 



VARIABIE lUEAN STANDARD MMIMUM MAXIMUM STD ERROR 





S0C3RE 


DEVIATION 


SfJQRE 




CFMEAN 


01 


4.76 


0.66 


2.00 


5.00 


0.09 


02 


4.36 


1.19 


0.00 


5.00 


0.17 


03 


4.30 


1.30 


0.00 


5.00 


0.18 


04 


4.50 


1.09 


0.00 


5.00 


0.15 


05 


4.32 


0.91 


2.00 


5.00 


C.13 


06 


4.44 


0.88 


2.00 


5.00 


0.13 


07 


4.40 


1.05 


0.00 


5.00 


0.15 


08 


4.10 


1.27 


0.00 


5.00 


0.18 


09 


4.40 


1.05 


0.00 


5.00 


0.15 


01 0 


4.04 


1.51 


0.00 


5.00 


0.21 


Oil 


4.34 


1.21 


0.00 


5.00 


0.17 


012 


4.74 


0.80 


0.00 


5.00 


0.11 


013 


4.38 


1.40 


0.00 


5.00 


0.20 


014 


4.24 


1.48 


0.00 


5.00 


0.21 


015 


4.74 


0.63 


3.00 


5.00 


0.09 


016 


4.70 


0.74 


2.00 


5.00 


0.10 


017 


3.74 


1.78 


0.00 


5.00 


0.25 


018 


4.66 


0.94 


1.00 


5.00 


0.13 


019 


4.70 


0.74 


2.00 


5.00 


0.10 


O20 


4.38 


1.12 


1.00 


5.00 


0.16 


021 


4.54 


1.16 


0.00 


5.00 


0.16 


022 


2.52 


2.36 


0.00 


5.00 


0.33 


SUM20 


88.24 


10.88 


60.00 


100.00 


1.54 


SUM22 


95.30 


12.43 


65.00 


110.00 


1.76 
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Table C3. 

Mean, standard deviation, and mininTum and maximum scores by the two peer graders for individual 
Hems and total test tor the 18 preservice social studies teachers in Group C. 



VARIABUE WEM STAhtDARD MINIMUM MAXIMUM STD ERROR 





SfiORF 


DEVIATON 




score 


OF MEAN 


01 


4.28 


0.94 


2.00 


5.00 


0.16 


02 


4.17 


0.85 


2.00 


5.00 


0.14 


03 


4.31 


0.95 


1.00 


5.00 


0.16 


04 


4.47 


1.06 


0.00 


5.00 


0.18 


Q5 


4.56 


0.81 


2.00 


5.00 


0.13 


06 


4.53 


0.74 


3.00 


5.00 


0.12 


07 


4.39 


0.80 


3.00 


5.00 


0.13 


08 


4.44 


0.88 


2.00 


5.00 


0.15 


09 


4.25 


0.87 


2.00 


5.00 


0.15 


01 0 


4.31 


0.89 


2.00 


5.00 


0.15 


011 


3.78 


1.24 


0.00 


5.00 


0.21 


012 


4.50 


0.74 


3.00 


5.00 


0.12 


013 


4.36 


0.99 


0.00 


5.00 


0.17 


014 


3.44 


2.06 


0.00 


5.00 


0.34 


015 


4.58 


0.94 


1.00 


5.00 


0.16 


016 


4.75 


0.60 


3.00 


5.00 


0.10 


017 


3.39 


1.79 


0.00 


5.00 


0.30 


018 


4.47 


1.23 


0.00 


5.00 


0.21 


019 


3.89 


1.09 


0.00 


5.00 


0.18 


O20 


4.00 


1.33 


0.00 


5.00 


0.22 


021 


3.92 


1.57 


0.00 


5.00 


0.26 


022 


3.31 




0.00 


5.00 


0.33 


SUM20 


84.86 


8.60 


67.00 


99.00 


1.43 


SUM22 


92.08 


10.24 


67.00 


109.00 


1.71 
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Table C4. 

Mean, standard deviation, and minimum and maximum scores by the ooer graders lor individual 
items and total test for the 22 preservice social studies teachers in Group C . 



VARIABLE VEAN STANDARD M^ilMUM MAXIMUM STD ERROR 







DEVIATION 




SCORE 


OF MEAN 


W i 




0 75 


2 00 


5 00 


0.11 


02 


•».«?■» 


1 24 


0 00 

W» WW 


5.00 


0.19 


Wa 


4 <;0 

•♦.31/ 


1 D2 


0 00 


5 00 


0 15 




4 48 


1 13 


0 00 


5 00 


0.17 




4 34 


0 89 


1 00 


5.00 


0.13 




4 05 


1 31 


0 00 


5.00 


0.20 


Q7 


4 25 


1 06 


1 00 


5.00 


0.16 


OS 


4 25 


1 04 


0 00 


5.00 


0.16 




4 32 


1 12 


1 00 


9.00 


0.17 




4 25 


1.14 


0.00 


5.00 


0.17 




3 61 


1.81 


0.00 


5.00 


0.27 


Q12 


4.84 


0.48 


3.00 


5.00 


0.07 


013 


4.34 


1.52 


0.00 


5.00 


0.23 


014 


4.00 


1.64 


0.00 


5.00 


0.25 


015 


4.68 


0.80 


1.00 


5.00 


0.12 


016 


4.68 


0.91 


1.00 


5.00 


0.14 


017 


3.75 


1.88 


0.00 


5.00 


0.28 


018 


4.25 


1.46 


0.00 


5.00 


0.22 


019 


4.39 


0.81 


2.00 


5.00 


0.12 


020 


4.50 


1.00 


1.00 


5.00 


0.15 


021 


4.20 


1.72 


0.00 


5.00 


0.26 


022 


3.11 


2.17 


0.00 


5.00 


0.33 


SUM20 


86.48 


15.29 


22.00 


100.00 


2.31 


SUM22 


93.80 


17.63 


22.00 


110.00 


2.66 
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Table C5. 

Mean, standard deviation, and n^nimum and maximum scores by the two peer praders tor individual 
items and total test for the 6 preservice social studies teachers in Group E. 



VARIABLE MEAN STANDARD MINiMtUM MAXlMtUM STD ERROR 







DEVIATtON 


SCORE 


SOORE 


OF MEAN 


01 


5 00 


0 00 


5 00 


5.00 


0.00 


02 


4 94 


0 25 


4 00 


5.00 


0.06 




4 94 


0 25 


4.00 


5.00 


0.06 




4 8S 


0 50 


3 00 


5.00 

W • w w 


0.12 




4 £9 


0 70 


3 00 


5.00 


0.18 


WD 


4 69 


0 48 


4 00 


5 00 


0.12 


07 


4 50 


1 03 


1 00 


5.00 


0.26 


Oft 


4 63 


0 89 


2 00 


5 00 


0.22 


OQ 


4 00 


1 86 


0 00 


5 00 


0.47 


W i V 


4 50 


0.97 


2.00 


5.00 


0.24 


oil 


4.38 


1.41 


0.00 


5.00 


0.35 


012 


4.88 


0.50 


3.00 


5.00 


0.12 


01 3 


4.94 


0.25 


4.00 


5.00 


0.06 


014 


4.50 


1.10 


2.00 


5,00 


0.27 


015 


4.88 


0.34 


4.00 


5.00 


0.09 


016 


4.94 


0.25 


4.00 


5.00 


0.06 


017 


4.06 


1.34 


1.00 


5.00 


0.34 


018 


4.50 


0.97 


2.00 


5.00 


0.24 


019 


4.69 


0.60 


3.00 


5.00 


0.15 


020 


4.69 


0.60 


3.00 


5.00 


0.15 


021 


4.75 


0.45 


4.00 


5.00 


0.11 


022 


3.50 


2.10 


0.00 


5.00 


0.52 


SUM20 


93.19 


6.73 


76.00 


100.00 


1.68 


SUM22 


101.44 


7.92 


82.00 


110.00 


1.98 
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Table C6. 

Mean, standard deviation, and maximum scores by the two peer graders for the individual items and 
total test for the 84 preservice social studies teachers in the five methods classes comtiined. 



VARIABLE 



01 

02 

03 

C4 

05 

06 

07 

08 

09 

01 0 

011 

012 

013 

014 

015 

018 

017 

018 

019 

O20 

021 

022 

SUM20 

SUM22 



MEAN 

4.58 
4.29 
4.42 
4.48 
4.44 
4.39 
4.35 
4.23 
4.27 
4.25 
3.94 
4.63 
4.37 
3.98 
4.63 
4.72 
3.74 
4.45 
4.36 
4.35 
4.27 
3.10 
86.86 
94.23 



STANDAFS) 
PgV'ATtON 

0.82 

1.07 

1.03 

1.03 

0.84 

0.94 

0.95 

1.06 

1.12 

1.16 

1.43 

C.81 

1.31 

1.64 

0.81 

0.71 

1.67 

1.15 

0.88 

1.07 

1.36 

2.10 
11.50 
13.10 



MMIMUM 
SCORE 

2.00 
0.00 
0.00 
0.00 
1.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
1.00 
1.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
22.00 
22.00 



MAXIMUM 
SCOT 

5.00 
5.00 
5.00 
5.00 
5.00 
5.00 
5.00 
5.00 
9.00 
5.00 
5.00 
5.00 
5.00 
5.00 
5.00 
5.00 
5.00 
5.00 
5.00 
5.00 
5.00 
5.00 
100.00 
110.00 



STD ERROR 
OF MEAN 

0.06 
0.08 
0.08 
0.08 
0.06 
0.07 
0.07 
0.08 
0.09 
0.09 
0.11 
0.06 
0.10 
0.13 
0.06 
0.05 
0.13 
0.09 
0.07 
0.08 
0.11 
0.16 
0.89 
1.01 
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Table C7. 

Mean, standard deviation, and minimum and n^imum scores by the two peer graders for the 
individual items and total test for the 29 secondary social studies students. 



VARIABLE tJEAN STANDARD MINIMUM MAXIMUM STDEmOR 







DEVIATION 




SCOP£. 


OF MEAN 


W 1 


A 71 


0 70 


2 no 

. WW 


5 00 


0 09 

W » Wm 






1 19 

1 . 1 £ 


0 00 

w.ww 


5 00 

•/ • WW 


0 15 








V - WW 


5 00 


0 16 






1 08 

1 •WW 


0 00 


5 00 


0 14 


OS 




0 91 


2 00 


5 00 


0.12 




4 47 


0 7B 


2 00 


5 00 


0 10 


07 


4 


0 99 


0 00 


5 00 


0 13 


OR 


4 O'i 


1 21 


0 00 


5 00 

*/ • WW 


0 16 

W • ■ V 


00 


4 ')4 


1 02 


0 00 

Vr • W 


5 00 


0 13 


OlO 


4 12 


1 39 


0 00 


5 00 


0.18 


Q11 


4.21 


1.22 


0.00 


5.00 


0.16 


012 


4.64 


0.87 


0.00 


5.00 


0.11 


01 3 


4.40 


1.34 


0.00 


5.00 


0.18 


014 


4.14 


1.46 


0.00 


5.00 


0.19 


015 


4.67 


0.69 


3.00 


5.00 


0.09 


016 


4.67 


0.73 


2.00 


5.00 


0.10 


017 


3.78 


1.68 


0.00 


5.00 


0.22 


018 


4.55 


0.96 


1.00 


5.00 


0.13 


019 


4.52 


0.80 


2.00 


5.00 


0.11 


Q20 


4.29 


1.08 


1.00 


5.00 


0.14 


021 


4.55 


0.94 


0.00 


5.00 


0.12 


022 


2,83 


2.23 


0.00 


5.00 


0.29 


SUM20 


87.29 


11.12 


60.00 


100.00 


1.46 


SUM22 


94.67 


12.27 


65.00 


110.00 


1.61 
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Table C8. 

Means, standard deviations, minimum and maximum scores for ditierences In scores on the same 
exam by two peer secondary school sodal studies students who graded the same examination.* 



VARtABt.E MEAN STANDARD iWlR^IMUt^ MAXIMUM STD ERROR 
ntFFFRFNCg Pg VtATlONI DFFPRPNCF DIFFgRENCE OF MEAN 



DQ1 


0.59 


0.91 


0.00 


3.00 


0.17 


DQ2 


0.66 


0.77 


0.00 


2.00 


0.14 


DC3 


0.66 


0.81 


0.00 


3.00 


0.15 


004 


0.62 


1.24 


0.00 


5.00 


0.23 


DOS 


0.90 


0.82 


0.00 


3.00 


M5 


DOS 


0.79 


0.77 


0.00 


2.00 


0.14 


007 


0.62 


0.78 


0.00 


3.00 


0.14 


008 


0.97 


0.91 


0.00 


3.0O 


0.17 


DQ9 


0.83 


1.20 


0.00 


5.00 


0.22 


001 0 


0.79 


0.86 


0.00 


3.00 


0.16 


0011 


0.97 


1.27 


0.00 


5.0O 


0.24 


0012 


0.52 


0.95 


0.00 


4.0O 


0.18 


0013 


0.38 


0.62 


0.00 


2.00 


0.12 


0014 


0.76 


0.99 


0.00 


4.00 


0.18 


0015 


0.38 


0.73 


0.00 


2.00 


0.14 


0016 


0.66 


0.94 


0.00 


3.00 


0.17 


0017 


1.00 


1.22 


0.00 


5.00 


0.23 


0018 


0.55 


0.91 


0.00 


4.00 


0.17 


0019 


0.48 


0.91 


0.00 


3.00 


0.17 


DO20 


0.66 


1.04 


0.00 


4.00 


0.19 


0021 


0.34 


0.55 


0.00 


2.00 


0.10 


0022 


0.90 


1.37 


0.00 


5.00 


0.25 


OSUM20 


9.97 


8.61 


0.00 


34.00 


1.60 


0SUM22 


10.45 


8.47 


0.00 


34.00 


..57 


SMABOF2013.76 


8.75 


2.00 


38.00 


1.62 


SMABDF2215.00 


9.19 


2.00 


40.00 


1.71 



•N = 29 students. 
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Table D1. 

Copy of the form used to collect definitions from the participants. 

Defining Terms 

Listed below are a number of terms that are commonly found on assignments, tests, and 
materials used in the classroom. After each word, write a definition of what that term means 
you need to write down to get a correct answer. Tor example, an item on a test may ask you "to 
describe (or explain) the Boston Tea Party." In this situation, what would it mean for you"to 
describe?" or "to explain?" 

Please take vour time and define each term as completely as possible^ What vou write 
down may help us to design more worthwhile assignments and better test questions. 

1. Identify 

2. Describe 

3. Explain 

4. Understand 

5. Comprehend 

6. Compare 

7. Contrast 

8. Define 
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9 . Analyze 



10. Discuss 



11. Cause 
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Table D2. 

A sample of ihe definilions the pre- and in-senicc teachers provided for each of 

the eleven terms ihcy were asked to define following their grading of the take 
home exam. 



IdgntifY ^ . ^ ^. ^ 

pinpoln? persorvplacethlng and state what part they played in what is being studied 
~ noting specific features 

- to pick out parts of a story or characters 

to name or state who someone is or wha: something is 

- to catagorize 

-- to categorize things 

- to know and explain 

- to show that you know what it is 

list the attributes of 

- express somethir^ or something 

explaining the setting and structure and plot of what happened as completely as possibt<» 
to give details which tells atx>ut someone, something or an event 

- explEuning the details of items or ideas 

- give in detail 

- to explain in terms of appearances, characteristics of, etc. 

- to Hst and explain specific details about a specific item 
show understanding by explaining what it is 

Explain 

-- using the attributes to show osmprehension 

- discuss and write about a situation 

- discuss whatever happened as completely as possible 

- to give details, "reasons" to describe a concept 

- disoissing the meaning or reason 

- discuss the meaning of something 

- to describe 

to state in one's own words details about an event, object, or ac. 
discuss or write the meaning or reason for something 

Understand 

- mostly deals with facts 

- comprehending and knowledge of material 

- to be able to talk and write about the story or whatever easily 

~ to find meaning in a concept. You learn what something means. 

- comprehend 

- too similar to comprehend. Just have to use one or the other. 

- to comprehend 

~ to have in one's own mind a good picture of a particular concept. 

- to have a good idea about what they are doir^ 
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CftinprPhgnd 

~ understand the reasons behirK) the facts 

- recalling the understanding of material 

- unde^ar%i n^terial that was read 

- you are able to go a step further from understanding. You can apply a concept, you can compare 
and contrast, find relationships, etc. 

- understand 

- too similar to understarxi. Just have to use one or the other. 

- to understand 

- to understand completely 

~ to understand the meaning of 

Compare 

- mostly listir^ opposite and the same attributes 

- take two situations and what is different or the same 

- write differences between two different things explaining each 

- to find similarities between two concepts, find their relationships 

- the similarities of 2 or more Items, or concepts 
~ similarities 

- to Ust the sirrtilarities 

- to fist likenesses 

- describe the sintilarities of two or more things 
Contrast 

- view of the opposite points of a specific topic 

- same as above for conpare 

-- describe the difference to the item or the opposite of it 

- to find differences between two concepts and find their relationships 
the differences between 2 or n^re items or concepts 

- differences 

to list the differences 
to list differences 

describe the differences between two or more items 

- how is a specific word or phrase used in the context of what has been studied 

- educated meaning 

" a statement describing something 

" to give the definition or tell what a word or concept means 

- the Bterary nwaning of a word or concept 

- brief explanation 
" to give meaning to 

~ state characteristics and specifics 

-- to give the nfieaning of a word or phrase 
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- deive into the reasons - interpretations of tacts 

- discuss a situation 

- oroarUzed thougtits of occurrances arui your opinion of then> 

- to take a "in deptti Jook* at}Out a concept, find fault or good, you often Incorporate your beliefs and 
opinions imo analysis. 

making an ot^rvation and trying to come to a corn^sion 
-tell what you tfiir* 

- to break down into secttons and discuss 
~ to corrpare, contrast, and summarize 

» ttiink over an object, idea, and come to a hypothesis 

Pisfuss 

- converse or write in nanative form what was learned atx>ut the topic 

- talk or wTtte about a situation 

answer the question as best you know and add or comment to the statement at hand 

- to answer questions, find relationships, cause-effect, consequences, sequence, etc. Cover all 
aspects of a concept so you fully understand it. 

-- \d3k about certain concepte, values, ideas, t^ellefs. etc. 

- to talk about in detail 

- taB( about in general temns 

-- talk about an objeci, idea give rrwny different views 

Clause 

- the reasons/feefings/happenings behind an action or happening 

- explain why sonrething happened 

- why an occun-ence happened 

- to state why sorrothing happened has occuned, events that led up to cause "if to happen 
-- the reason for an event har^ning 

- events leading to cause something 
-- the purpose 

- reasons for the way something is 



